Migration support for bit indexed explicit replication

ABSTRACT

A method and network device are disclosed for multicast forwarding. In one embodiment, the method includes receiving at a node configured to use a bit-indexed forwarding table a multicast message comprising an incoming message bit array. The method further includes comparing the incoming message bit array to a first neighbor bit array within a first forwarding table entry, and determining that for at least one relative bit position in the bit arrays a corresponding destination node is both a destination for the message and a reachable destination from a first neighboring node. The method further includes forwarding toward the first neighboring node a copy of the message, where a first node on the path of the forwarded message is not configured to use a respective bit-indexed forwarding table. An embodiment of the device includes a network interface, a memory and a processor configured to perform steps of the method.

RELATED APPLICATIONS

This application is a continuation-in-part of U.S. application Ser. No.14/488,790, entitled “Bit Indexed Explicit Replication UsingMultiprotocol Label Switching,” filed Sep. 17, 2014, which in turnclaims the domestic benefit under Title 35 of the United States Code§119(e) of U.S. Provisional Patent Application Nos. 61/878,693, entitled“Multicast IPv6 with Bit Mask Forwarding,” filed Sep. 17, 2013, and61/931,473, entitled “Bit Mask Forwarding Architectures for StatelessMultipoint Replication,” filed Jan. 24, 2014. This application is also acontinuation-in-part of U.S. application Ser. No. 14/488,761, entitled“Bit Indexed Explicit Replication,” which in turn claims the domesticbenefit under Title 35 of the United States Code §119(e) of U.S.Provisional Patent Application Nos. 61/878,693, entitled “Multicast IPv6with Bit Mask Forwarding,” filed Sep. 17, 2013, and 61/931,473, entitled“Bit Mask Forwarding Architectures for Stateless MultipointReplication,” filed Jan. 24, 2014. This application is also acontinuation-in-part of U.S. application Ser. No. 14/488,810, entitled“Bit Indexed Explicit Replication Using Internet Protocol Version 6,”filed Sep. 17, 2014, which in turn claims the domestic benefit underTitle 35 of the United States Code §119(e) of U.S. Provisional PatentApplication Nos. 61/878,693, entitled “Multicast IPv6 with Bit MaskForwarding,” filed Sep. 17, 2013, and 61/931,473, entitled “Bit MaskForwarding Architectures for Stateless Multipoint Replication,” filedJan. 24, 2014. Each of the two provisional and three non-provisionalapplications referenced above is hereby incorporated by reference in itsentirety and for all purposes as if completely and fully set forthherein.

BACKGROUND

Network nodes forward data. Network nodes may take form in one or morerouters, one or more bridges, one or more switches, one or more servers,or any other suitable communications processing device. The data iscommonly formatted as messages and forwarded using forwarding tables. Amessage is a formatted unit of data that typically contains controlinformation and payload data. Control information may includeinformation that identifies sources and destinations, such as addresses,error detection codes like checksums, sequencing information, etc.Control information is typically found in message headers and trailers.Payload data is typically located between the message headers andtrailers. Depending on factors such as the network level and networkprotocol used, a message may be formatted and/or referred to as one ofvarious specific types such as packets, datagrams, segments, or frames.

Forwarding messages involves various processes that, while simple inconcept, can be complex. The processes involved in forwarding vary,depending on the type of forwarding method used. Overall forwardingconfigurations include unicast, broadcast, and multicast forwarding.Unicast is a method of point-to-point communication most often used whena particular node (known as a source) wishes to send data to anotherparticular node (known as a receiver) and is not concerned with sendingthe data to multiple receivers. Broadcast is method used when a sourcewishes to send data to all receivers in a domain, and multicast allows asource to send data to a group of receivers in a domain while preventingthe data from being sent to other receivers in the domain.

Multicast is the preferred method of data forwarding for many popularapplications, such as streaming media distribution. One reason for thisis that multicast is a bandwidth-conserving technology that allowsdelivery of data to multiple receivers while avoiding transmission ofmultiple copies of the same message over the same network link. However,in traditional multicast systems, a relatively large amount of controlplane information is used. Setting up and maintaining this controlinformation has a tendency to become complex and costly in terms ofcomputing resources, and can become a major limiting factor in overallnetwork performance.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure may be better understood, and its numerousobjects, features, and advantages made apparent to those skilled in theart by referencing the accompanying drawings.

FIG. 1A is a simplified block diagram illustrating certain components ofan example network.

FIG. 1B is a simplified block diagram illustrating certain components ofan exemplary network device that may be associated with one of the nodesof the network of FIG. 1A.

FIG. 2A is a simplified block diagram illustrating certain components ofan example network.

FIG. 2B is a simplified block diagram illustrating certain aspects of anexemplary forwarding process using the network of FIG. 2A.

FIG. 3 is a simplified block diagram illustrating certain components ofan example network.

FIG. 4A is an exemplary routing table generated by a node of the networkof FIG. 3.

FIG. 4B is an exemplary routing table generated by a node of the networkof FIG. 3.

FIG. 5A is an exemplary forwarding table generated by a node of thenetwork of FIG. 3.

FIG. 5B is an exemplary forwarding table generated by a node of thenetwork of FIG. 3.

FIG. 6 is a simplified block diagram illustrating certain components ofan example network.

FIG. 7A is an exemplary routing table generated by a node of the networkof FIG. 6.

FIG. 7B is an exemplary forwarding table generated by a node of thenetwork of FIG. 6.

FIG. 8 is a flow chart illustrating an example process employed by anode of FIG. 6.

FIG. 9 is a flow chart illustrating an example process employed by anode of FIG. 6.

FIG. 10A is a flow chart illustrating an example process employed by anode of FIG. 6.

FIG. 10B is an exemplary routing table generated by a node of thenetwork of FIG. 6.

FIG. 10C is an exemplary forwarding table generated by a node of thenetwork of FIG. 6.

FIG. 11 is a block diagram illustrating certain components of an examplenetwork device that can be employed in the networks described herein.

FIG. 12 is a block diagram depicting a computer system suitable forimplementing embodiments of the devices and systems described herein.

DETAILED DESCRIPTION Overview

A method and network device are disclosed for multicast forwardingthrough bit indexed explicit replication (BIER). In one embodiment, themethod includes receiving at a node in a network a multicast messagecomprising an incoming message bit array, where the node is configuredto store and use a bit-indexed forwarding table comprising multipleforwarding table entries. Each of the multiple forwarding table entriescomprises in this embodiment a respective neighbor bit array and isassociated with a respective neighboring node, and a set of multiplepossible destination nodes for the message corresponds to the same setof respective relative bit positions in the incoming message bit arrayand each of the neighbor bit arrays. In this embodiment, the methodfurther includes comparing the incoming message bit array to a firstneighbor bit array associated with a first forwarding table entry of themultiple forwarding table entries and determining that, for at least onerelative bit position the corresponding destination node is both anintended destination for the message and a reachable destination from afirst neighboring node associated with the first forwarding table entry.The method further includes forwarding toward the first neighboring nodea copy of the message comprising a forwarded message bit array, where afirst node on the path of the forwarded message is not configured to usea respective bit-indexed forwarding table.

Multicast

Multicast transmission delivers multicast data packets (data packetsthat traditionally include information identifying a multicast group,such as a multicast group address) from a source to multiple receiverswithout unduly burdening the source. Although much of the discussion inthis disclosure is in terms of packets, it should be understood that thedisclosures made herein may also be applicable to other types of networkmessages, such as datagrams or data frames. As used herein, the term“receiver” signifies a host (such as a computing device or application)that has subscribed to a multicast group. Instead of the sourcereplicating a multicast data packet and sending a copy of the multicastdata packet to each receiver, the source sends a single copy of amulticast data packet and multicast-enabled routers (referred to hereinsimply as nodes) replicate the packet at the point(s) where paths tovarious receivers diverge. Multicast routing protocols enable multicasttransmission (i.e., one-to-many connections and many-to-manyconnections) by replicating a multicast data packet close to thedestination of that multicast data packet, obviating the use of multipleunicast connections for the same purpose. This saves network bandwidthand improves throughput.

FIG. 1A is a simplified block diagram of a network 100 performingmulticast data transmission. Multicast-enabled nodes 110, 120, 130 and140 are coupled through network links 150, 160, and 170.Multicast-enabled node 110 is also coupled to source 111 and receiver112; multicast-enabled node 120 is coupled to receiver 121;multicast-enabled node 130 is coupled to receiver 131 and receiver 132;and multicast-enabled node 140 is coupled to receiver 141. Such couplingbetween the multicast-enabled nodes and the sources and/or receivers canbe direct or indirect (e.g., via a L2 network device or another node).

For the purposes of this illustration, source 111 is a host configuredto transmit multicast data packets to a multicast group that includes asreceivers hosts 112, 121, 131, 132 and 141. Source 111 transmits amulticast flow, consisting of one or more multicast data packets havinga common multicast group address, to multicast-enabled node 110(illustrated by the arrow from 111 to 110). Multicast-enabled node 110includes a multicast forwarding table that multicast-enabled node 110uses to determine where to forward the multicast data packets associatedwith the multicast flow. The multicast forwarding table includesinformation identifying each interface of multicast-enabled node 110that is connected via a path in the form of a multicast distributiontree (MDT) to one or more receivers for the multicast group (e.g., ahost that has sent a join message, as described above).Multicast-enabled node 110 then replicates multicast data packets in themulticast flow and transmits the replicated multicast data packets fromthe identified interfaces to receiver 112, multicast-enabled node 120,and multicast-enabled node 130.

Multicast-enabled nodes 120 and 130 inform node 110 that they arecoupled to one or more receivers using join messages such as, forexample, a protocol independent multicast (PIM) join message. Inresponse to receiving the join messages, multicast-enabled node 110updates its multicast forwarding tables to identify interfaces to whichmulticast data packets should be forwarded. The multicast data packetscan be replicated by node 110, and then nodes 130 and 120, as needed inorder to provide the multicast data packets to receivers for themulticast group (e.g., receivers 131 and 132) and othermulticast-enabled nodes on the MDT (e.g., multicast-enabled node 140).In this manner, a multicast flow from source 111 can be transmittedthrough a multicast network to multiple receivers.

A block diagram of an exemplary network device that may be associatedwith a node in network 100 of FIG. 1A is shown in FIG. 1B. Networkdevice 150 of FIG. 1B may, for example, be associated withmulticast-enabled node 120 in FIG. 1A. In some cases “node” as usedherein encompasses a network device associated with the node. “Networkdevices” as used herein includes various devices, such as routers,switches, or network controllers, that perform routing functions andsupport a routing protocol. A network device maintains one or morerouting tables that stores routing information identifying routes tovarious data sources and/or data consumers. In a multicast-enabled node,a network device implements a multicast routing protocol that is used toconvey multicast data packets from a multicast source such as source 111of FIG. 1A to a multicast receiver. For each multicast group to which amulticast source sends data, the multicast routing protocol canestablish a multicast distribution tree, which is a group of couplednodes that can convey packets from the multicast source to the multicastreceivers.

In the embodiment of FIG. 1B, network device 150 includes storage formulticast routing information 152, storage for multicast forwardinginformation 164, a routing module 160, and an interface 162. Interface162 is coupled to send and receive packets. It is noted that networkdevice 150 may include additional interfaces, and that each interfacecan be a logical or physical interface.

Routing module 160 is configured to perform multicast routing based onthe stored multicast routing information 152. Routing module 160 is alsoconfigured to update the stored multicast forwarding information 164. Aforwarding engine (not shown) can forward multicast data packets usingthe stored multicast forwarding information 164. Routing module 160 canimplement one or more instances of a unicast routing protocol and one ormore instances of a multicast routing protocol.

Entry 170 provides an example of the routing information that can bestored for a particular multicast group. As shown, entry 170 includes asource address (S) 154, a group address (G) 156, and reverse pathforwarding (RPF) identifying information (“RPF info”) 158. The RPFidentifying information identifies which interface within a networkdevice associated with node 110 properly receives multicast data packetsaddressed to group G, as well as the RPF neighbor that is properlyforwarded those multicast data packets. The RPF interface is theinterface leading to the root of the multicast tree for group G (e.g.,the root of the multicast tree can be the rendezvous point associatedwith group G). The storage for multicast routing information 152 is, inone embodiment, implemented as a Multicast Routing Information Base(MRIB).

Entry 172 provides an example of the forwarding information that can bestored for a particular multicast group. As shown, entry 172 includes asource address (S) 154, a group address (G) 156, an incoming interface(IIF) list 166, and an outgoing interface (OIF) list 168. A forwardingengine uses the information in entry 172 to forward multicast datapackets addressed to multicast group G. For example, when a packethaving destination address G is received, the forwarding engine accessesentry 172 and verifies the source address and incoming interface of thepacket. If the packet was received via an interface other than the oneidentified in IIF 166, the packet is dropped. If the packet matches theinformation in entry 172, the packet is forwarded from the interfaceslisted in OIF 168. The storage for multicast forwarding information 164is, in one embodiment, implemented as a Multicast Forwarding InformationBase (MFIB).

The above-described process traditionally used in setting up MDTs andupdating multicast forwarding tables for each multicast group results inconsiderable amounts of state information within the network. Themulticast forwarding tables maintained by each multicast-enabled node,in particular, can become quite large in networks with many sources,many groups, or both. Maintaining such multicast forwarding tablesrepresents limitations on network scalability.

Bit Indexed Explicit Replication

As described herein, the amount of state information within a multicastnetwork may be reduced by methods, devices and systems in which receiverinformation is carried by the packet rather than being looked up intables at each network node based on source and group information. In anembodiment, a group of receivers is represented by an array of bitscarried in a packet, and the packet is forwarded based on this receiverinformation. This greatly reduces the amount of state information storedat nodes and is therefore also referred to as “stateless multicast.”More formally, the term Bit Indexed Explicit Replication (BIER) is usedto describe this approach. As suggested by the term, a bit position isused as an index into a forwarding table and packets are replicated onlyto specified nodes.

FIG. 2 shows an example network 200. Network 200 includes BIER-enablednodes 206-218. BIER-enabled nodes are configured to forward packetsusing BIER. For example, BIER-enabled nodes are configured to store anduse bit-indexed forwarding tables, as explained further below.BIER-enabled nodes 206-218 form a provider network, or domain. Such aprovider network could be employed by an Internet service provider totransport packets to customers. The domain includes core nodes 208 and210, and provider edge nodes 206, 214, 216, and 218. The provider edgenodes are coupled to customer edge nodes 211, 213, 215, and 217. Hosts201, 203, 205, and 207 are coupled to the customer edge nodes. In theembodiment of FIG. 2, host 201 is a multicast source, while hosts 203,205 and 207 are configured as multicast receivers, or subscribers.

Each of the BIER-enabled nodes 206-218 has interfaces that areidentified as shown. For example, BIER-enabled node 208 has threeinterfaces designated 1-3, respectively. Each BIER-enabled node isassigned a unique identifier or routable address known as a routeridentifier (RID). The RID can be implemented as, for example, aninternet protocol (IP) address, prefix, or loopback address. Network 200and the other BIER-enabled networks described herein are not limited toany particular version of IP, or to any particular routing protocol atall. Each BIER-enabled node advertises or floods the routable address toall other BIER-enabled nodes in network 200. Each BIER-enabled nodebuilds a unicast topology of the BIER-enabled nodes in network 200 usingthe advertised routable addresses.

BIER-enabled node 206 is configured as an ingress router for multicastdata packets. The ingress router is coupled, via customer edge node 211,to source 201. Multicast data packets from source 201 enter the BIERnetwork via the ingress router (BIER-enabled node 206). Each ofBIER-enabled nodes 214, 216, and 218 is configured as an egress router.The egress routers can be connected (directly or via customer edgerouters) to hosts, such as receivers, or other networks. An egressrouter is a BIER-enabled node that is the last BIER-enabled node on apath between a source and a receiver. As such, an egress router is adestination node when forwarding using BIER. The egress router may be aprovider edge node that is coupled to the receiver either directly orindirectly (e.g., through a non-BIER-enabled customer edge node).

In an embodiment, receiver information is included in the packet byassigning each egress router in a BIER network a bit position (BP)within a packet bit array carried by the packet (or, more generally, amessage bit array carried by a network message). An egress router isalso associated with the same relative bit position in a neighbor bitarray stored in a bit-indexed forwarding table at a BIER-enabled node.Each of the packet bit array and neighbor bit array may also be referredto as a bit mask (BM) herein. In some embodiments, the packet bit arrayis referred to as a bit string or BitString and the neighbor bit arrayis referred to as a bit mask. As used herein, the term bit array, bitstring or bit mask refers to a set of bits that has a fixed or variablelength. The length of the bit arrays used in a particular BIER networkcan be statically configured or dynamically assigned and distributedthrough the BIER network. The bit array can have any suitable length. Inan embodiment, the length is determined in view of the size andcapabilities of the network. In one embodiment, the length of the bitarray is between 8 and 4096 bits. In a further embodiment, the length ofthe bit array is between 256 and 1024 bits. The maximum bit array lengthvalue is determined, in one embodiment, by hardware or softwarelimitations of the BIER-enabled nodes in the BIER network. In anembodiment, the number of egress routers represented by a bit array isincreased by including a set identifier in the packet along with the bitarray. The same bit position can then be used to represent one egressrouter in, for example, set 0, and a different egress router in set 1.In some embodiments, sets are used for network management purposes suchas multi-topology routing, temporal slicing, or grouping ofgeographically-proximate nodes.

A bit position can be statically or dynamically assigned to an edgerouter. The bit position may be assigned by a central authority, such asa network controller, or through another mechanism such as derivation ofa BP from an identifier for the router. Each edge router should have atleast one unique bit position within the bit array. In an embodiment,multiple BPs are assigned to a single edge router, to allow multicastdelivery, for example, to multiple receivers connected to the edgerouter via separate interfaces of the router. The edge router (orinterface) associated with a bit position may vary with time in someembodiments, for purposes such as failure response or optimization ofnetwork performance.

BIER Packet Forwarding Example

To illustrate the operation of BIER packet forwarding, network 200 ofFIG. 2A is shown again with additional annotation in FIG. 2B. In theembodiment of FIG. 2B, BIER-enabled node 214 (an egress router) signalsto BIER-enabled node 206 (an ingress router) that BIER-enabled node 214is interested in receiving packets associated with a given multicastgroup or flow. BIER-enabled node 216 likewise signals BIER-enabled node206 that BIER-enabled node 216 is interested in the same multicastgroup. The signaling is represented by the dashed lines shown in FIG. 2.BIER-enabled node 206 updates an entry in group membership table (GMT)224 (or creates one if one does not already exist) for the multicastgroup and updates a packet bit array (PBA) in the entry by setting bitscorresponding to BIER-enabled nodes 214 and 216. Assuming that onlyBIER-enabled nodes 214 and 216 are interested in the flow, and notBIER-enabled node 218, the PBA is {0101}.

In the simplified example of FIG. 2B, the packet bit array and neighborbit arrays used are four bits long, which is sufficient to represent thethree egress routers in network 200, each connected to a respective oneof the three receivers in the network. In this example, a “1” value in abit position of a packet bit array indicates that the correspondingdestination node is an intended destination for the packet. Analternative convention for the value at a bit position could be used inanother embodiment, but in any case the value of the bit at a bitposition in a packet bit array indicates whether the correspondingdestination node is an intended destination.

BIER-enabled node (and ingress router) 206 is configured to receive amulticast data packet 234 addressed to the multicast group or flow(e.g., from source 201 via customer edge node 211). BIER-enabled node206 uses the multicast group address and/or source address included inthe multicast data packet to access its GMT and select a packet bitarray associated with the multicast group. After selecting a PBA thatcorresponds to the multicast group from the GMT, BIER-enabled node 206encapsulates the packet bit array into the multicast data packet,resulting in BIER packet 236. Ingress node 206 also identifies theneighbors to which packet 236 will be forwarded. In an embodiment, theneighbors are identified using the bit-indexed forwarding table (BIFT)of node 206, a portion 226 of which is shown in FIG. 2B. In a furtherembodiment, this involves performing an AND operation between the packetbit array and each neighbor bit array (NBA) in BIER-enabled node 206'sBIFT. In this example, there is only one entry in the BIFT and the entrycorresponds to BIER-enabled node 208. This means that the shortest pathfrom BIER-enabled node 206 to all three of the egress routers in network200 runs through BIER-enabled node 208. Since the result of the AND isTRUE for neighbor B (BIER-enabled node 208), BIER-enabled node 206forwards the multicast data packet to BIER-enabled node 208. Thisforwarding may involve other information from the BIFT for node 206 notshown in portion 226, such as egress interface information. In theembodiment of FIG. 2B, BIER-enabled node 206 also modifies the packetbit array in the multicast data packet it forwards, as discussed furtherbelow.

In response to receiving the multicast data packet, BIER-enabled node208 performs an AND between the packet bit array in the multicast datapacket, {0101}, and the neighbor bit array in each entry in its BIFT (aportion 228 of which is shown). The result for neighbor C is TRUE soBIER-enabled node 208 forwards the multicast data packet to BIER-enablednode 210. BIER-enabled node 208 also modifies the packet bit array inthe multicast data packet it forwards, as discussed below. The resultfor neighbor E is also TRUE, so BIER-enabled node 208 replicates themulticast data packet and forwards the multicast data packet toBIER-enabled node 216, which is an egress router. In the example of FIG.2B, a “1” value in a bit position of a neighbor bit array indicates thatthe destination node assigned to the bit position is reachable from theneighboring node corresponding to the forwarding table entry containingthe neighbor bit array. An alternative convention for the value at a bitposition could be used in another embodiment, but in any case the valueof the bit at a bit position in a neighbor bit array indicates whetherthe corresponding destination node is a reachable destination from theneighbor associated with the neighbor bit array.

BIER-enabled node 210, in response to receiving a copy of the multicastdata packet, performs an AND between the packet bit array in themulticast data packet, {0001}, and the neighbor bit array in each entryin its BIFT (portion 230 of which is shown). The result for neighbor Dis TRUE so BIER-enabled node 210 forwards the multicast data packet toBIER-enabled node 214 which is an egress router. The result for neighborF is FALSE, so BIER-enabled node 210 refrains from forwarding themulticast data packet to BIER-enabled node 218. In this way themulticast data packet travels from the ingress router (BIER-enabled node206) through the BIER network to the two egress routers that signaled aninterest in the multicast group (BIER-enabled nodes 214 and 216).

In the embodiment of FIG. 2B, each time the BIER packet is forwardedusing an entry in a bit-indexed forwarding table, the packet bit arrayin the forwarded packet is altered to clear any set bits in bitpositions corresponding to nodes not reachable from the neighbor thatthe packet is being forwarded to. For example, when the multicast packetarrives at node B, it has an incoming packet bit array of {0101}.Comparison of the packet bit array to the neighbor bit arrays shown inBIFT portion 228 shows that the set first (rightmost) bit of the PBAcorresponds to a destination node reachable through neighbor C, whilethe set third bit corresponds to a node reachable through neighbor E.The packet bit array in the packet forwarded to neighbor C accordinglyhas only the first bit set, and the PBA in the packet forwarded toneighbor E has only the third bit set. This modification of the packetbit array when a BIER packet is forwarded prevents looping andduplication by ensuring that a BIER-enabled node forwards a givenmulticast data packet only once based on a given bit position. Thisalteration of the packet bit array to clear bits that are not also setin the neighbor bit array is a form of masking by the neighbor bitarray. In an alternative embodiment, the above-described modification ofthe packet bit array can be done as the packet arrives at the next noderather than as it leaves the forwarding node.

In addition to alteration of the packet bit array sent with a forwardedpacket (which may also be called a forwarded packet bit array herein),the packet bit array used at a BIER-enabled node for comparison to eachneighbor bit array within a BIFT may be modified each time a packet issent. Specifically, if a packet is sent as a result of comparing theincoming PBA to a neighbor bit array in a bit-indexed forwarding tableat the node, the PBA used for comparison to the next neighbor bit arrayin the forwarding table is altered to remove the destinations of thejust-sent packet as intended destinations. Returning to the operation ofnode B in FIG. 2B, in one embodiment the incoming PBA of {0101} iscompared to NBA {0011} for neighbor C. Because bit position 1 is set inboth of these arrays, a packet is sent to neighbor C with the PBAmodified in the sent packet as described above. In addition, theincoming PBA may be altered so that position 1 is no longer set beforemoving down the table to compare to NBA {0100} for neighbor E. The PBAused to compare to the forwarding table entry for neighbor E istherefore {0100} in such an embodiment. Because position 1 is not set inthe NBA for neighbor E anyway, alteration of the PBA before comparisondoes not have an effect in this case. This alteration can preventsending of a duplicate packet, however, in a case for which multipleforwarding table entries have an NBA with the same bit set. This canhappen, for example, in equal cost multi-path (ECMP) arrangements.

The above-described modifications to the packet bit array are not neededin embodiments in which the network has a loop-free topology. Oneexample of a loop-free topology is a point-to-multipoint (P2MP) labelswitched path (LSP) in a network employing multiprotocol label switching(MPLS). Modifications to the packet bit array may also be omitted inembodiments in which some amount of looping and/or duplication can betolerated.

Bit-Indexed Routing and Forwarding Tables

Each BIER-enabled node in the BIER network uses the BPs and routeridentifiers (RIDs) of the other BIER-enabled nodes to generate one ormore bit-indexed routing tables (BIRTs) and bit-indexed forwardingtables (BIFTs). A bit-indexed routing table is a table that storesBP-to-router identifier mappings. In an embodiment, the BIER-enablednodes learn about the BP-to-router ID mappings through advertisementssent by the BIER-enabled nodes having assigned bit positions.

In response to a BP being assigned to an egress router, the egressrouter advertises its BP along with its router identifier to some or allof the other nodes in the BIER network. In one embodiment, the ERadvertises its BP via an interior gateway protocol (IGP). Within anautonomous system, an IGP is used for exchanging network topologyinformation between nodes (all nodes, whether BIER-enabled or not). Anautonomous system, or routing domain, as used herein refers to acollection of interconnected network nodes under a common administrationfor purposes of network configuration. There are different types ofIGPs, which vary in terms of, for example, the particular informationexchanged between nodes, whether information is shared only withneighbor nodes or “flooded” throughout the autonomous system, and howoften the exchanged information is updated. In one type of IGP called alink-state routing protocol, every router constructs a topological mapof network connectivity in the form of a graph, showing which routersare connected to which other routers. Each router can use its map toindependently calculate the best logical path from it to every possibledestination in the network. The collection of best paths will then formthe routing table. Examples of link-state routing protocols include theintermediate system to intermediate system (IS-IS) and the Open ShortestPath First (OSPF) protocols. Messages called advertisements are used inIGPs to exchange information. Nodes in an IP network automaticallyexchange network topology information through IGP advertisements.

In an embodiment, ISIS and/or OSPF protocols can be modified to assistin distributing BP-to-router ID mappings through the BIER network usinglink state updates. In OSPF, such a link state update is called alink-state advertisement (LSA). Certain types of LSAs are “opaque” LSAswhich are forwarded through the network even by nodes that do notthemselves have the capability to use the information in the LSA. Suchopaque LSAs may be useful in networks having both BIER-enabled andnon-BIER enabled nodes, as noted below in the discussion of migration toBIER-enabled networks. Other flooding mechanisms to distribute theinformation are possible. All BIER-enabled nodes in a BIER network, notjust the egress routers, also flood their respective router identifiers,which are used in building network topology and unicast forwardingtables. BIER-enabled nodes, in one embodiment, advertise additionalinformation as well, such as a bit mask size that the BIER-enabled nodeis configured to use. Adding such BIER information to the advertisedinformation is a relatively small amount of additional information, ascompared with the usual topology information exchanged through IGPadvertisements, and the state information maintained on a per-groupbasis in traditional multicast.

Using a mechanism such as IGP advertisements, each BIER-enabled nodereceives BP-to-router identifier mappings and stores them in a BIRT. Inan embodiment using an MPLS implementation of BIER, the BIER-enablednode also includes at least one label range in the BIRT for each routerID. If multiple bit array sizes are in use, BIER-enabled nodes advertisemultiple label ranges, for example, one label range for each bit arraysize.

Using the router identifiers, a BIER-enabled node performs a recursivelookup in unicast routing tables to identify a directly connected nexthop BIER-enabled node (referred to herein as a neighbor (Nbr)) on theshortest path from the BIER-enabled node toward the BIER-enabled nodeassociated with the BP, and the interface via which the neighbor isreachable. In one embodiment, the neighbor is the next hop on a shortestpath (SPT) towards the egress router that originated the advertisementof the bit position. In one embodiment, the BIRT includes one entry perBP. In an MPLS implementation, each entry can include multiple labelranges associated with the router ID; for example, if the BIER-enablednode uses multiple bit array sizes, each bit array size has anassociated label range.

Example BIRTs and BIFTs are described in the context of FIG. 3. FIG. 3is similar to FIG. 2A, in that FIG. 3 depicts an example network 300.Network 300 includes BIER-enabled nodes 306-318. BIER-enabled nodes306-318 form a provider network, or domain. Such a provider networkcould be employed by an Internet service provider to transport packetsto customers. The domain includes core nodes 308 and 310, and provideredge nodes 306, 314, 316, and 318. Each BIER-enabled node is assignedand advertises the router ID (RID) shown. In the case of ingress router306 and egress routers 314, 316, and 318, the set identifier and bitposition shown are also assigned and advertised. In the embodiment ofFIG. 3, network 300 implements MPLS, so that each BIER-enabled node alsoallocates and advertises the label range initial value shown.

BIER-enabled node 316 is shown as being assigned a BP in set 1, whileBIER-enabled nodes 306, 314, and 318 are in set 0. In an embodiment,each BIER-enabled node has allocated a label range such that each sethas its own label. If BIER-enabled node 306 forwards a multicast datapacket having one or more bits set at respective BPs corresponding toegress routers in set 0, BIER-enabled node 306 uses label 20. IfBIER-enabled node 306 forwards a multicast data packet having one ormore bits set for BPs corresponding to egress routers in set 1,BIER-enabled node 306 uses label 21. In this example there are two sets(0 and 1). If there were a third set—e.g., an ER had been assigned a BPin set 2—BIER-enabled node 306 would use label 22 for BPs correspondingto egress routers in set 2. In one embodiment, each BIER-enabled nodeallocates a contiguous range of labels, e.g., 256 labels, to accommodatea set length used by some or all of the BIER-enabled nodes in thenetwork. In the example of FIG. 3, we assume the bit array length is thesame on all BIER-enabled nodes.

Using the example MPLS-implemented BIER network of FIG. 3, FIGS. 4A and4B show exemplary BIRTs constructed by BIER-enabled nodes 306 and 308,respectively. As shown in FIG. 4A, BIER-enabled node 306 constructs abit-indexed routing table 400. Bit-indexed routing table 400 includes acolumn 402 for router ID received in advertisements from other networknodes. The router ID, in one embodiment, is a prefix assigned to eachnode. BIRT 400 also includes a column 404 for information identifyingthe set and bit position associated with the BIER-enabled egress nodesidentified in the router ID column. In an embodiment, interior nodesthat are neither ingress nor egress nodes, such as the nodes labeled Band C, are not assigned bit positions. Nodes B and C therefore have nullentries in column 404. At 406, bit-indexed routing table 400 includes acolumn for information identifying a label range associated with (e.g.,advertised by) each of the BIER-enabled nodes identified in the routerID column. In the embodiment of FIG. 4A, the initial label in the rangeis provided.

Bit-indexed routing table 400 also includes, at 408, a column for theneighbor used for routing to each node in the table. The neighbor columnidentifies the BIER-enabled router that is next on a path between node306 and the node identified in the RID column of the bit-indexed routingtable. For example, as shown in FIG. 3, the next hop BIER-enabled nodebetween BIER-enabled node 306 (A/32) and BIER-enabled node 314 (D/32),is BIER-enabled node 308 (B/32). Bit-indexed routing table 400 may alsoinclude other information not shown in FIG. 4A, such as egress interfaceinformation and other information that might also appear in atraditional routing table.

FIG. 4B shows a bit-indexed routing table for BIER-enabled node 308.Bit-indexed routing table 420 is similar to BIRT 400, and accordinglyincludes router ID column 402, set:bit position column 404, label rangecolumn 406 and neighbor column 408 as described for table 400 above. Thevalues within these columns are different from those in thecorresponding columns of table 400, since table 420 is for use by node308 (labeled “B” in FIG. 3) rather than node 306 (labeled “A” in FIG.3).

Each BIER-enabled node translates its BIRT(s) into one or morebit-indexed forwarding tables (BIFTs). FIG. 5A shows an exemplarybit-indexed forwarding table 540. In one embodiment, BIFT 540 is createdby BIER-enabled node 306 of FIG. 3. BIFT 540 includes column 542, whichincludes information identifying local, or incoming, MPLS labels used bythe BIER-enabled node that created the BIFT (BIER-enabled node 306 inthe example of FIG. 5A.) The local label column 542 includes a locallabel for each set in which bit positions have been assigned. In thecase of FIG. 5A, set 0 (which corresponds to local label 20) and set 1(which corresponds to local label 21) are in use by BIER-enabled node306. Table 540 also includes a bit position column 544. For each locallabel, each bit position that has been assigned to an egress routerreachable from the node using table 540 has an entry in the embodimentof FIG. 5A. Column 546 includes information identifying a neighbor bitarray (NBA) which can be compared to a packet bit array within amulticast data packet arriving at BIER-enabled node 306. To the extentthat any reachable nodes indicated by the neighbor bit array are alsointended destination nodes for the arriving multicast packet (indicatedin this example by set bits in the packet bit array), a forwarded packetbit array matching the neighbor bit array is sent with the forwardedmulticast data packets toward the egress routers corresponding to thebits indicated in the neighbor bit array. At 548, informationidentifying the neighbor along the shortest path towards the egressrouter corresponding to the BP identified in column 544 is included.

BIFT 540 also includes a remote, or outgoing, label column 550. TheBIER-enabled node that maintains the forwarding table, in this case,BIER-enabled node 306, inserts the label identified in column 550 into amulticast data packet's header and then forwards the multicast datapacket along the shortest path towards the neighbor identified in thecorresponding BIFT entry. Bit-indexed forwarding table 540 may alsoinclude other information not shown in FIG. 5A, such as egress interfaceinformation and other information that might also appear in atraditional forwarding table.

Similar to bit-indexed forwarding table 540 of FIG. 5A, bit-indexedforwarding table 560 of FIG. 5B includes information used by aBIER-enabled node to forward multicast data packets. In one embodiment,BIFT 560 is created by BIER-enabled node 308 of FIG. 4. Bit-indexedforwarding table 560 includes local label column 542, bit positioncolumn 544, neighbor bit array column 546, neighbor column 548, andremote label column 550, as described above for BIFT 540. The valueswithin these columns are different from those of table 540, however,since table 560 is configured for forwarding from a different node.

In the example of FIGS. 4B and 5B, in response to receiving a multicastdata packet with local label 30 (corresponding to set 0), a BIER-enablednode determines how to forward the packet. Since egress routerscorresponding to bit position 1 (SI:BP equal to 0:1) and bit position 2(SI:BP equal to 0:2) are shown in routing table 420 of FIG. 4B to bereachable via C, the corresponding BPs are aggregated to form neighborbit array 0011 in forwarding table 560, which the BIER-enabled node putsin the BIFT entries corresponding to neighbor C. The aggregationinvolves, in one embodiment, performing a logical OR operation betweenbit arrays that each have a bit set only in the BP corresponding to therespective egress router reachable from the neighbor. The egress routercorresponding to bit position 3 (SI:BP equal to 0:3) is shown in routingtable 420 to be reachable via A. The corresponding bit is set in theneighbor bit array for neighbor A in BIFT 560. For set 1 (label 31), theER corresponding to bit position 1 (SI:BP equal to 1:1) is shown inrouting table 420 to be reachable via E.

Migration to a BIER-Enabled Network

The above discussion of BIER network operation is generally in thecontext of networks in which all nodes between (and including) theingress node and egress node are BIER-enabled (e.g., able to accesspacket bit arrays and to create and use bit-indexed forwarding androuting tables). Because practical networks are often expanded, upgradedand otherwise reconfigured over time, however, capabilities of the nodesand their associated network devices can be inhomogeneous across anetwork. An upgrade requiring only a software download and install atone node may require replacement of line cards or other hardware atanother node. A practical network configured to use BIER for multicasttransmission may nonetheless include some nodes that are notBIER-enabled.

Such a situation is illustrated by the exemplary network shown in FIG.6. Network 600 of FIG. 6 is almost identical to network 300 of FIG. 3,except that in network 600 the node labeled B (and having a router ID ofB/32) is implemented by non-BIER-enabled node 602 (the lack of BIERcapability is indicated by cross-hatching of the representation of thenode in FIG. 6). In the embodiment of FIG. 6, the router ID of B/32 fornode 602 is the same type of ID used by non-BIER-enabled nodes innetwork 600, and would therefore be advertised by node 602 as part ofthe normal configuration and updating process employed by a networkusing an IGP. In the MPLS network embodiment of FIG. 6, node 602 willalso advertise MPLS labels, but it will not advertise a label range suchas the range beginning with label 30 shown in FIG. 3, because the labelranges of FIG. 3 are allocated as BIER labels. Use of a label within theranges of FIG. 3 is one way to alert a BIER-enabled node that the packetcarrying the label is a BIER multicast packet. In an embodiment,specific labels corresponding to particular label switched paths (LSPs)and particular network interfaces of node 602 will be advertised. Thelabels advertised by node 602 will be from a range not allocated toBIER; in an embodiment, the labels are exchanged with neighboring nodesusing label distribution protocol (LDP). As such, nodes of network 600not directly connected to node 602 may not receive any informationregarding labels for node 602.

As a non-BIER-enabled node, node 602 stores typical unicast routing andforwarding tables but does not store a bit-indexed routing table formulticast such as table 420 in FIG. 4B or a bit-indexed forwarding tablesuch as table 560 in FIG. 5B. Moreover, the entries related to neighborB in the BIRTs and BIFTs of the BIER-enabled nodes within network 600reflect that neighbor B is not BIER enabled. FIG. 7A shows an exemplarybit-indexed routing table 700 for BIER-enabled node 306 of network 600in FIG. 6. Table 700 is similar to table 400 of FIG. 4A, except in thecase of entry 702 for the router with router ID B/32. In an embodiment,the advertisements sent by the BIER-enabled nodes of network 600 aresent using opaque link-state advertisements (LSAs). In such anembodiment, a non-BIER-enabled node such as node 602 will forwardBIER-related LSAs that it receives on to other nodes, even though thenon-enabled node can't itself understand these LSAs. Flooding of theBIER-related LSAs by all network nodes allows a BIER-enabled node suchas node 306 to form a bit-indexed routing table lacking only entryinformation for the non-BIER-enabled nodes. As noted in the discussionof FIG. 6 above, non-BIER-enabled node 602 does not advertise aBIER-related label range; the label range is therefore missing fromentry 702.

Similar information is missing from the exemplary bit-indexed forwardingtable shown in FIG. 7B for node 306 of network 600. Because paths to theegress routers can be determined through use of unicast routing tables,node 306 can determine which egress routers are reachable from neighborB even though that node is not BIER-enabled. But because noBIER-specific label range is advertised by node 602, the remote label incolumn 550 of table 710 will need to be a non-BIER-related label, suchas a standard unicast label. The unicast label needed depends on theparticular LSP used to forward a packet to node B—i.e., how far thepacket is being sent. Some specific approaches to forwarding of a BIERpacket through a non-BIER-enabled node are provided below.

One embodiment of a method for forwarding a BIER packet in a networkincluding a non-BIER-enabled node is shown in the flowchart of FIG. 8.In an embodiment, the method of FIG. 8 is performed by a network deviceassociated with a BIER-enabled node. The method begins with receiving aBIER multicast packet at step 802. The packet bit array within thepacket is compared to a neighbor bit array in a forwarding table entryfor a neighbor node (step 804). The forwarding table in step 804 is abit-indexed forwarding table comprising an entry with a neighbor bitarray, such as forwarding table 540 of FIG. 5A or forwarding table 710of FIG. 7B. In an embodiment, the forwarding table entry also includes abit position, such as the bit positions in column 404 of FIG. 4A orcolumn 544 of FIG. 7B. The packet bit array is compared to the neighborbit array to determine whether any destination nodes for the packet arealso reachable nodes from the neighbor associated with the forwardingtable entry (decision step 806). An egress router for the network is anexample of a destination node for a packet. In an embodiment, if a bitis set in the same relative bit position in both the packet bit arrayand the neighbor bit array, the destination node is both a destinationnode for the packet and a reachable node from the neighbor.

In some embodiments, the comparison of the packet bit array and neighborbit array to determine whether a destination node for the packet is areachable node via the neighbor is done by considering one bit at a timeof the packet bit array and neighbor bit array. In such an embodiment,the forwarding table may include an entry for each bit positioncorresponding to a reachable destination node, and the entries may besorted by bit position, as shown, for example, by table 540 of FIG. 5Aor table 710 of FIG. 7B. The comparison in such an embodiment mayinclude checking one bit at a time of the packet array until a bitposition with a set bit is found, then looking in a bit position columnof the forwarding table for an entry with a set bit in the same bitposition. Such a bit-by-bit approach may be faster in some embodimentsthan working through the forwarding table one neighbor at a time.Sorting the forwarding table entries by bit position may also beadvantageous in helping to identify nodes having two different neighbornodes with a shortest path to the same destination node. In such asituation, an equal cost multi-path (ECMP) can be applied to choose oneof the neighbor nodes for forwarding.

If no destination node of the packet is also a reachable node via aneighbor associated with a forwarding table entry (N branch of decision806), the method of FIG. 8 checks whether any bits in the packet bitarray are still set (decision step 808). If no bits are set, there areno multicast destinations remaining for the packet, and the method ends.If there are still one or more bits set in the packet bit array, themethod checks whether any additional neighbor nodes are included in theforwarding table entries (decision step 810). If there are no additionalneighbor nodes represented in the table (that have not already beenchecked by comparison of the packet bit array and neighbor bit array),the method ends. A situation in which set bits remain in the packet bitarray but no neighbor nodes for forwarding remain in the forwardingtable may represent, for example, a failure cause by a change in networkconfiguration or a mislabeled packet. If there is both a set bitremaining in the packet bit array and a neighbor node to be checked forreachable destinations, the method continues by selecting a forwardingtable entry for the next neighbor node in the table (step 812), andcomparison of the packet bit array to the neighbor bit array for the newforwarding table entry (step 804).

When comparison of the packet bit array and a neighbor bit array for aforwarding table entry reveals an intended destination node for thepacket that is also reachable through the neighbor node associated withthe forwarding table entry, it is determined whether the neighbor nodeis BIER-enabled (step 814). In an embodiment, IGP advertisements areused to determine whether a node is BIER-enabled. For example, an opaqueLSA may be established to advertise BIER capability throughout thenetwork. As another example, the IGP neighbor hello message may beextended to include a capability field advertising BIER capabilityand/or BIER mask size support. In an embodiment employing MPLS, aBIER-enabled node performing the method of FIG. 8 can check itsbit-indexed routing table for the label range advertised by the neighbornode to see whether a label range allocated for BIER has been advertisedby the neighbor.

If the neighbor node is BIER-enabled, a copy of the packet is forwardedto the neighbor node (step 818). In the embodiment of FIG. 8, the packetbit array of the forwarded packet is altered to form a forwarded packetbit array. In the forwarded packet bit array, any set bits in theincoming packet bit array in bit positions not corresponding toreachable destinations via the neighbor node are cleared. In otherwords, for any destination nodes that were indicated in the incoming PBAas intended destinations but are not reachable via the neighbor node,the forwarded PBA is altered to indicate that those destinations are notintended destinations. In step 820 of the method of FIG. 8, analteration is also made to the version of the packet bit array used forcomparison with the next forwarding table entry. To create a “comparisonPBA” that is compared to the neighbor bit array in the next forwardingtable entry, set bits in the current packet bit array in bit positionscorresponding to those reachable by the just-forwarded packet arecleared in the comparison packet bit array. The comparison packet bitarray is then used as the packet bit array in subsequent steps of themethod. If there are still bits set in the comparison PBA (step 808),and if there are more neighbor nodes in the forwarding table (step 810),the method continues with the comparison packet bit array used for thenext comparison to a neighbor bit array. The packet bit arrayalterations of steps 818 and 820 are optionally employed to preventlooping and duplication of packets. These alterations may be omitted inembodiments for which duplication and looping are not present or areotherwise not of concern.

The above-described steps of the method of FIG. 8 represent anembodiment of a BIER multicast forwarding method when the forwardingnode and the node forwarded to are both BIER-enabled. In a practicalnetwork, a BIER-enabled node may have some BIER-enabled neighbors towhich replicated packets may be forwarded in the manner described above.Other neighbors of the BIER-enabled node may be non-BIER-enabled,however, and one embodiment of a method for forwarding to these nodesstarts with the “N” branch of decision step 814.

In the embodiment of FIG. 8, if it determined that a neighbor nodehaving reachable destination nodes is not BIER-enabled, there is first adetermination whether there is an alternate BIER-enabled neighbor (step816). In other words, is there another, BIER-enabled neighbor node thatone or more of the same destination nodes is reachable from? If so, thealternate neighbor is used instead in the embodiment of FIG. 8, and theBIER forwarding process described above is carried out. In anotherembodiment, algorithms for selecting ECMP paths are configured to favorpaths with BIER-enabled nodes if possible. In a further embodiment, suchalgorithms are configured to favor paths with BIER-enabled nodes if thenumber of packets being replicated is large enough that a significanttraffic benefit is realized by using a BIER-enabled path.

If the neighbor node is not BIER-enabled and there is no alternateBIER-enabled neighbor (“N” branches of decision steps 814 and 816), themulticast packet is replicated to provide a separate copy of the packetfor each of the destination nodes reachable via the non-BIER-enabledneighbor node (step 822). For each destination node, the respectivepacket copy includes BIER encapsulation for that destination node (step824), and the packet copy is forwarded to the destination node byunicast forwarding (step 826). In an embodiment, the BIER encapsulationfor the destination node includes a packet bit array having the bit inthe bit position corresponding to the destination node set. In a furtherembodiment having an MPLS implementation, the BIER encapsulationincludes a BIER-specific MPLS label assigned to the destination nodethat is inserted into the packet below a unicast MPLS label assigned tothe non-BIER-enabled neighbor node. A “tunnel” to the BIER-enableddestination node is thereby created. In other embodiments a tunnel to aBIER-enabled node is created using a different tunneling mechanism, suchas IPv6 Generic Routing Encapsulation (GRE) tunneling. When a packet hasbeen forwarded for each destination node reachable via thenon-BIER-enabled neighbor (“N” branch of decision step 828), the methodreturns to step 820 where the packet bit array is modified to unset bitscorresponding to destination nodes that packets have been forwarded to.From there, if there are set bits remaining in the packet bit array andentries for additional neighbor nodes remaining in the bit-indexedforwarding table, the process continues with the next neighbor node.

In the method of FIG. 8, when it is determined that a neighbor node isnot BIER-enabled, the packet is replicated for each of the destinationnodes reachable through the non-BIER enabled node, and the packet copiesare sent to the respective destination nodes by unicast forwarding. Adifferent approach to forwarding via a non-BIER-enabled node is shown inthe method of FIG. 9. Steps 902 through 918 of FIG. 9 are similar tosteps 802 through 820 of FIG. 8 in implementing forwarding using aBIER-enabled neighbor node. The flowchart of FIG. 9 omits thedetermination of whether a BIER-enabled alternative node exists for anon-BIER-enabled neighbor node; this determination can be included inother embodiments of the method.

When one or more intended destination nodes for a BIER multicast packetare reachable via a non-BIER-enabled neighbor node (“N” branch ofdecision step 914), the BIER-enabled forwarding node determines thenearest BIER-enabled node along a the shortest path to each reachabledestination node (step 920). In an embodiment, determination of thenearest BIER enabled node is done by performing a recursive lookup inunicast routing tables to identify the next-hop nodes between thenon-BIER-enabled neighbor node and the destination nodes. The forwardingnode determines whether the next-hop nodes are BIER-enabled or not, in amanner similar to that discussed above for step 814 in FIG. 8. If any ofthe next-hop nodes are also non-BIER-enabled, the recursive lookupprocess is repeated to identify further next-hop nodes along the pathsto any of the destination nodes, until a BIER-enabled node is found foreach destination node reachable from the non-BIER-enabled neighbor node.The multicast packet is replicated once for each of the identifiednearest BIER-enabled nodes (step 922). Assuming that multipledestination nodes are reachable via each of at least some of theidentified nearest BIER-enabled nodes, the method of FIG. 9 preservessome multicast routing efficiency by replicating separate packet copiesonly for each of the nearest BIER-enabled nodes rather than for eachdestination node as is done in the method of FIG. 8.

In step 922, the forwarding node further keeps track of whichdestination nodes are reachable via which of the identified nearestBIER-enabled nodes, and creates a forwarded packet bit array for eachreplicated packet having bits set at bit positions corresponding to thedestination nodes reachable via the identified nearest BIER-enabled nodefor that packet copy. BIER encapsulation for the identified nearestBIER-enabled node is also included in the packet copy for that node(step 924), and the packet copy is forwarded to the nearest BIER-enablednode by unicast forwarding (step 926). BIER encapsulation may beprovided in a manner similar to that described for step 824 in FIG. 8.In an embodiment, the use of unicast forwarding to the nearestBIER-enabled node with underlying BIER encapsulation provides a “tunnel”for the packet to the BIER-enabled node. When a packet has beenforwarded for each identified nearest BIER-enabled node on a path to adestination node reachable via the non-BIER-enabled neighbor (“N” branchof decision step 928), the method returns to step 918 where the packetbit array is modified to unset bits corresponding to destination nodesthat packets have been forwarded to. From there, if there are set bitsremaining in the packet bit array and entries for additional neighbornodes remaining in the bit-indexed forwarding table, the processcontinues with the next neighbor node.

An additional approach to BIER forwarding via non-BIER-enabled nodes isshown in FIGS. 10A through 10C. The flowchart of FIG. 10A has the formof a method for forwarding in a BIER-enabled network, and does notinclude a determination of whether the neighbor node is BIER-enabled.This is a result of incorporation of the tunneling described in themethods of FIG. 8 and FIG. 9 above into the routing and forwardingtables associated with the BIER-enabled node performing the method ofFIG. 10.

An exemplary bit-indexed routing table 1018 for node 306 in network 600of FIG. 6 is shown in FIG. 10B. Routing table 1018 includes onlyBIER-enabled nodes of network 600, and therefore does not include routerID B/32 for node 602. Moreover, neighbor column 1026 does not includeneighbor B as the directly-connected neighbor on the path between node306 and each network node, but instead includes the nearest BIER-enablednode between node 306 and each of the other BIER-enabled nodes innetwork 600. In an embodiment, the nearest BIER-enabled nodes aredetermined using recursive lookup in unicast routing tables, in themanner discussed above for step 920 in FIG. 9. The columns in table 1018are otherwise similar to those of the bit-indexed routing tables ofFIGS. 4A, 4B, and 7A.

Bit-indexed forwarding table 1040 of FIG. 10C is generated by node 306using routing table 1018. Like routing table 1018 of FIG. 10B,forwarding table 1040 does not include node 602 having router ID B/32 asa neighbor, instead using BIER-enabled neighbors C and E in neighborcolumn 1034. Remote label column 1036 accounts for the presence of node602, however. In addition to the BIER label 40 for neighbor C and BIERlabel 61 for neighbor E, forwarding table 1040 provides unicast labelsfor forwarding to non-BIER-enabled node 602. In the embodiment of FIG.10C, label 330 is the unicast MPLS label assigned to node 602 for theLSP terminating at neighbor C, and label 340 is the unicast MPLS labelassigned to node 602 for the LSP terminating at neighbor E. Forwardingusing table 1040 pushes both labels onto a BIER multicast packet andsends the packet to node 602. Node 602, as the penultimate node in eachLSP, removes the unicast label without replacing it. The packet thenarrives at the nearest BIER-enabled node with the appropriateBIER-allocated label so that BIER multicast forwarding can continue. Inthis way static tunnels are embedded into the bit-indexed forwardingtable.

FIG. 11 is a block diagram illustrating certain additional and/oralternative components of nodes that can be employed, for example in thenetworks shown in FIGS. 2, 3, and 6. In this depiction, node 1100includes a number of line cards (line cards 1102(1)-(N)) that arecommunicatively coupled to a forwarding engine or packet forwarder 1110and a processor 1120 via a data bus 1130 and a result bus 1140. Linecards 1102(1)-(N) include a number of port processors 1150(1,1)-(N,N)which are controlled by port processor controllers 1160(1)-(N). It willalso be noted that forwarding engine 1110 and processor 1120 are notonly coupled to one another via data bus 1130 and result bus 1140, butare also communicatively coupled to one another by a communications link1116.

The processors 1150 and 1160 of each line card 1102 may be mounted on asingle printed circuit board. When a packet or packet and header arereceived, the packet or packet and header may be identified and analyzedby router 1100 in the following manner. Upon receipt, a packet (or someor all of its control information) or packet and header is sent from theone of port processors 1150(1,1)-(N,N) at which the packet or packet andheader was received to one or more of those devices coupled to data bus1130 (e.g., others of port processors 1150(1,1)-(N,N), forwarding engine1110 and/or processor 1120). Handling of the packet or packet and headercan be determined, for example, by forwarding engine 1110. For example,forwarding engine 1110 may determine that the packet or packet andheader should be forwarded to one or more of port processors1150(1,1)-(N,N). This can be accomplished by indicating to correspondingone(s) of port processor controllers 1160(1)-(N) that the copy of thepacket or packet and header held in the given one(s) of port processors1150(1,1)-(N,N) should be forwarded to the appropriate one of portprocessors 1150(1,1)-(N,N). In addition, or alternatively, once a packetor packet and header has been identified for processing, forwardingengine 1110, processor 1120 or the like can be used to process thepacket or packet and header in some manner or add packet securityinformation, in order to secure the packet. On a node sourcing such apacket or packet and header, this processing can include, for example,encryption of some or all of the packet's or packet and header'sinformation, the addition of a digital signature or some otherinformation or processing capable of securing the packet or packet andheader. On a node receiving such a processed packet or packet andheader, the corresponding process is performed to recover or validatethe packet's or packet and header's information that has been thuslyprotected.

FIG. 12 is a block diagram of a computing device, illustrating how aforwarding module can be implemented in software, as described above.Computing system 1210 broadly represents any single or multi-processorcomputing device or system capable of executing computer-readableinstructions. Examples of computing system 1210 include, withoutlimitation, any one or more of a variety of devices includingworkstations, personal computers, laptops, client-side terminals,servers, distributed computing systems, handheld devices (e.g., personaldigital assistants and mobile phones), network appliances, switches,routers, storage controllers (e.g., array controllers, tape drivecontroller, or hard drive controller), and the like. In its most basicconfiguration, computing system 1210 may include at least one processor1214 and a system memory 1216. By executing the software that implementsa forwarding module 1217, computing system 1210 becomes a specialpurpose computing device that is configured to perform packetforwarding, in the manner described above.

Processor 1214 generally represents any type or form of processing unitcapable of processing data or interpreting and executing instructions.In certain embodiments, processor 1214 may receive instructions from asoftware application or module. These instructions may cause processor1214 to perform the functions of one or more of the embodimentsdescribed and/or illustrated herein. For example, processor 1214 mayperform and/or be a means for performing the operations describedherein. Processor 1214 may also perform and/or be a means for performingany other operations, methods, or processes described and/or illustratedherein.

System memory 1216 generally represents any type or form of volatile ornon-volatile storage device or medium capable of storing data and/orother computer-readable instructions. Examples of system memory 1216include, without limitation, random access memory (RAM), read onlymemory (ROM), flash memory, or any other suitable memory device.Although not required, in certain embodiments computing system 1210 mayinclude both a volatile memory unit (such as, for example, system memory1216) and a non-volatile storage device (such as, for example, primarystorage device 1232, as described in detail below). In one example,program instructions executable to implement a forwarding moduleconfigured to forward multicast data packets may be loaded into systemmemory 1216.

In certain embodiments, computing system 1210 may also include one ormore components or elements in addition to processor 1214 and systemmemory 1216. For example, as illustrated in FIG. 12, computing system1210 may include a memory controller 1218, an Input/Output (I/O)controller 1220, and a communication interface 1222, each of which maybe interconnected via a communication infrastructure 1212. Communicationinfrastructure 1214 generally represents any type or form ofinfrastructure capable of facilitating communication between one or morecomponents of a computing device. Examples of communicationinfrastructure 1214 include, without limitation, a communication bus(such as an Industry Standard Architecture (ISA), Peripheral ComponentInterconnect (PCI), PCI express (PCIe), or similar bus) and a network.

Memory controller 1218 generally represents any type or form of devicecapable of handling memory or data or controlling communication betweenone or more components of computing system 1210. For example, in certainembodiments memory controller 1218 may control communication betweenprocessor 1214, system memory 1216, and I/O controller 1220 viacommunication infrastructure 1214. In certain embodiments, memorycontroller 1218 may perform and/or be a means for performing, eitheralone or in combination with other elements, one or more of theoperations or features described and/or illustrated herein.

I/O controller 1220 generally represents any type or form of modulecapable of coordinating and/or controlling the input and outputfunctions of a computing device. For example, in certain embodiments I/Ocontroller 1220 may control or facilitate transfer of data between oneor more elements of computing system 1210, such as processor 1214,system memory 1216, communication interface 1222, display adapter 1226,input interface 1230, and storage interface 1234.

Communication interface 1222 broadly represents any type or form ofcommunication device or adapter capable of facilitating communicationbetween computing system 1210 and one or more additional devices. Forexample, in certain embodiments communication interface 1222 mayfacilitate communication between computing system 1210 and a private orpublic network including additional computing systems. Examples ofcommunication interface 1222 include, without limitation, a wirednetwork interface (such as a network interface card), a wireless networkinterface (such as a wireless network interface card), a modem, and anyother suitable interface. In at least one embodiment, communicationinterface 1222 may provide a direct connection to a remote server via adirect link to a network, such as the Internet. Communication interface1222 may also indirectly provide such a connection through, for example,a local area network (such as an Ethernet network), a personal areanetwork, a telephone or cable network, a cellular telephone connection,a satellite data connection, or any other suitable connection.

In certain embodiments, communication interface 1222 may also representa host adapter configured to facilitate communication between computingsystem 1210 and one or more additional network or storage devices via anexternal bus or communications channel. Examples of host adaptersinclude, without limitation, Small Computer System Interface (SCSI) hostadapters, Universal Serial Bus (USB) host adapters, Institute ofElectrical and Electronics Engineers (IEEE) 11054 host adapters, SerialAdvanced Technology Attachment (SATA) and external SATA (eSATA) hostadapters, Advanced Technology Attachment (ATA) and Parallel ATA (PATA)host adapters, Fibre Channel interface adapters, Ethernet adapters, orthe like.

Communication interface 1222 may also allow computing system 1210 toengage in distributed or remote computing. For example, communicationinterface 1222 may receive instructions from a remote device or sendinstructions to a remote device for execution.

As illustrated in FIG. 12, computing system 1210 may also include atleast one display device 1224 coupled to communication infrastructure1214 via a display adapter 1226. Display device 1224 generallyrepresents any type or form of device capable of visually displayinginformation forwarded by display adapter 1226. Similarly, displayadapter 1226 generally represents any type or form of device configuredto forward graphics, text, and other data from communicationinfrastructure 1214 (or from a frame buffer) for display on displaydevice 1224.

As illustrated in FIG. 12, computing system 1210 may also include atleast one input device 1228 coupled to communication infrastructure 1214via an input interface 1230. Input device 1228 generally represents anytype or form of input device capable of providing input, either computeror human generated, to computing system 1210. Examples of input device1228 include, without limitation, a keyboard, a pointing device, aspeech recognition device, or any other input device.

As illustrated in FIG. 12, computing system 1210 may also include aprimary storage device 1232 and a backup storage device 1233 coupled tocommunication infrastructure 1214 via a storage interface 1234. Storagedevices 1232 and 1233 generally represent any type or form of storagedevice or medium capable of storing data and/or other computer-readableinstructions. For example, storage devices 1232 and 1233 may be amagnetic disk drive (e.g., a so-called hard drive), a floppy disk drive,a magnetic tape drive, an optical disk drive, a flash drive, or thelike. Storage interface 1234 generally represents any type or form ofinterface or device for transferring data between storage devices 1232and 1233 and other components of computing system 1210. A storage devicelike primary storage device 1232 can store information such as routingtables and forwarding tables.

In certain embodiments, storage devices 1232 and 1233 may be configuredto read from and/or write to a removable storage unit configured tostore computer software, data, or other computer-readable information.Examples of suitable removable storage units include, withoutlimitation, a floppy disk, a magnetic tape, an optical disk, a flashmemory device, or the like. Storage devices 1232 and 1233 may alsoinclude other similar structures or devices for allowing computersoftware, data, or other computer-readable instructions to be loadedinto computing system 1210. For example, storage devices 1232 and 1233may be configured to read and write software, data, or othercomputer-readable information. Storage devices 1232 and 1233 may also bea part of computing system 1210 or may be a separate device accessedthrough other interface systems.

Many other devices or subsystems may be connected to computing system1210. Conversely, all of the components and devices illustrated in FIG.12 need not be present to practice the embodiments described and/orillustrated herein. The devices and subsystems referenced above may alsobe interconnected in different ways from that shown in FIG. 12.

Computing system 1210 may also employ any number of software, firmware,and/or hardware configurations. For example, one or more of theembodiments disclosed herein may be encoded as a computer program (alsoreferred to as computer software, software applications,computer-readable instructions, or computer control logic) on acomputer-readable storage medium. Examples of computer-readable storagemedia include magnetic-storage media (e.g., hard disk drives and floppydisks), optical-storage media (e.g., CD- or DVD-ROMs),electronic-storage media (e.g., solid-state drives and flash media), andthe like. Such computer programs can also be transferred to computingsystem 1210 for storage in memory via a network such as the Internet orupon a carrier medium.

The computer-readable medium containing the computer program may beloaded into computing system 1210. All or a portion of the computerprogram stored on the computer-readable medium may then be stored insystem memory 1216 and/or various portions of storage devices 1232 and1233. When executed by processor 1214, a computer program loaded intocomputing system 1210 may cause processor 1214 to perform and/or be ameans for performing the functions of one or more of the embodimentsdescribed and/or illustrated herein. Additionally or alternatively, oneor more of the embodiments described and/or illustrated herein may beimplemented in firmware and/or hardware. For example, computing system1210 may be configured as an application specific integrated circuit(ASIC) adapted to implement one or more of the embodiments disclosedherein.

Although the present disclosure includes several embodiments, thedisclosure is not intended to be limited to the specific forms set forthherein. On the contrary, it is intended to cover such alternatives,modifications, and equivalents as can be reasonably included within thescope defined by the appended claims.

What is claimed is:
 1. A method comprising: receiving at a node in anetwork a multicast message comprising an incoming message bit array,wherein the node is configured to store and use a bit-indexed forwardingtable comprising multiple forwarding table entries, each of the multipleforwarding table entries comprises a respective neighbor bit array andis associated with a respective neighboring node, and a set of multiplepossible destination nodes for the message corresponds to the same setof respective relative bit positions in the incoming message bit arrayand each of the neighbor bit arrays; comparing the incoming message bitarray to a first neighbor bit array associated with a first forwardingtable entry of the multiple forwarding table entries; determining thatfor at least one relative bit position the corresponding destinationnode is both an intended destination for the message and a reachabledestination from a first neighboring node associated with the firstforwarding table entry; and in response to said determining, forwardingtoward the first neighboring node a copy of the message comprising aforwarded message bit array, wherein a first node along the path of theforwarded message is not configured to use a respective bit-indexedforwarding table.
 2. The method of claim 1, wherein: a bit value of abit position in the message bit array indicates whether thecorresponding destination node is an intended destination for themessage; and a bit value of a bit position in each neighbor bit arrayindicates whether the corresponding destination node is reachable fromthe respective neighboring node associated with the forwarding tableentry.
 3. The method of claim 1, further comprising performing a firstmodification to the forwarded message bit array.
 4. The method of claim3, further comprising: in response to said determining, performing asecond modification to the incoming message bit array to form acomparison message bit array, wherein the comparison message bit arrayis configured to be compared to a second neighbor bit array associatedwith a second forwarding table entry of the multiple forwarding tableentries.
 5. The method of claim 3, wherein said performing a firstmodification comprises: for any destination node that is indicated bythe incoming message bit array as an intended destination but indicatedby the first neighbor bit array as an unreachable destination, alteringthe bit value of the corresponding relative bit position in theforwarded message bit array from that of the incoming message bit arrayto indicate that the destination node is not an intended destination. 6.The method of claim 4, wherein said performing a second modificationcomprises: for any destination node indicated by the incoming messagebit array as an intended destination and indicated by the first neighborbit array as a reachable destination, altering the bit value of thecorresponding relative bit position in the comparison message bit arrayfrom that of the incoming message bit array to indicate that thedestination node is not an intended destination.
 7. The method of claim1, wherein the multicast message further comprises a set identifierhaving multiple possible values; each possible value of the setidentifier corresponds to a different set of multiple possibledestination nodes for the message; and a combination of a set identifiervalue and a relative bit position in the incoming message bit arrayuniquely identifies a possible destination node for the message.
 8. Themethod of claim 1, wherein said comparing comprises: identifying arelative bit position in the incoming message bit array corresponding toan intended destination; and identifying an entry in the bit-indexedforwarding table for which the identified relative bit positioncorresponds to a reachable destination.
 9. The method of claim 1,wherein the multicast message comprises a multicast packet.
 10. Themethod of claim 1, wherein: The first node along the path of theforwarded message is the first neighboring node; and said forwardingcomprises sending a separate copy of the message to each of thedestination nodes determined to be both an intended destination for themessage and a reachable destination from the first neighboring node. 11.The method of claim 10, wherein said forwarding further comprisesaccessing a unicast forwarding table entry associated with eachdestination node that a copy of the message is forwarded to.
 12. Themethod of claim 11, wherein said forwarding further comprises includingwithin each forwarded message an identifier configured to alert therespective destination node to access the respective forwarded messagebit array.
 13. The method of claim 1, wherein the first node on the pathof the forwarded message is the first neighboring node, and saidforwarding comprises: identifying, for each of the destination nodesdetermined to be both an intended destination for the message and areachable destination from the first neighboring node, a nearest node,along a path to the respective destination node, that comprises arespective bit-indexed forwarding table; and sending a separate copy ofthe message to each of the identified nodes.
 14. The method of claim 13,wherein said sending comprises modifying the respective forwardedmessage bit array in each copy of the message to indicate as intendeddestination nodes only those destination nodes indicated as intended bythe incoming message bit array and also reachable from the respectiveidentified node that the copy of the message is sent to.
 15. The methodof claim 13, wherein said sending comprises including within eachforwarded message: a first identifier configured to be used by the firstnode on the path of the respective forwarded message in conveying themessage to the identified node along the path to a destination node forthe message; and a second identifier configured to alert the identifiednode to access the forwarded message bit array within the message. 16.The method of claim 1, wherein: the first neighboring node comprises arespective bit-indexed forwarding table; and the first forwarding tableentry comprises a first identifier configured to be included within theforwarded message and to be used by the first node on the path of theforwarded message in conveying the message to the first neighboringnode, and a second identifier configured to be included within theforwarded message and to alert the first neighboring node to access theforwarded message bit array.
 17. A network device associated with anetwork node, the network device comprising: one or more networkinterfaces; a memory configured to store a bit-indexed forwarding tablecomprising multiple forwarding table entries, wherein each of themultiple forwarding table entries comprises a respective neighbor bitarray and is associated with a respective neighboring node; and aprocessor configured to receive at the network node a multicast messagecomprising an incoming message bit array, wherein a set of multiplepossible destination nodes for the message corresponds to the same setof respective relative bit positions in the incoming message bit arrayand each of the neighbor bit arrays, compare the incoming message bitarray to a first neighbor bit array associated with a first forwardingtable entry of the multiple forwarding table entries, determine that forat least one relative bit position the corresponding destination node isboth an intended destination for the message and a reachable destinationfrom a first neighboring node associated with the first forwarding tableentry, and in response to a determination that an intended destinationfor the message is a reachable destination from the first neighboringnode, forward toward the first neighboring node a copy of the messagecomprising a forwarded message bit array, wherein a first node on thepath of the forwarded message does not comprise a respective bit-indexedforwarding table.
 18. The network device of claim 17, wherein: the firstnode on the path of the forwarded message is the first neighboring node;and the processor is further configured to send a separate copy of themessage to each of the destination nodes determined to be both anintended destination for the message and a reachable destination fromthe first neighboring node.
 19. The network device of claim 17, wherein:the first node on the path of the forwarded message is the firstneighboring node; and the processor is further configured to identify,for each of the destination nodes determined to be both an intendeddestination for the message and a reachable destination from the firstneighboring node, a nearest node, along a path to the respectivedestination node, that comprises a respective bit-indexed forwardingtable, and send a separate copy of the message to each of the identifiednodes.
 20. The network device of claim 17, wherein: the firstneighboring node comprises a respective bit-indexed forwarding table;and the first forwarding table entry comprises a first identifierconfigured to be included within the forwarded message and to be used bythe first node on the path of the forwarded message in conveying themessage to the first neighboring node, and a second identifierconfigured to be included within the forwarded message and to alert thefirst neighboring node to access the forwarded message bit array. 21.The network device of claim 20, wherein: the memory is furtherconfigured to store a bit-indexed routing table; and the routing tablecomprises entries only for nodes configured to use respectivebit-indexed forwarding tables.